Wisdom of Students: A Consistent Automatic Short Answer Grading Technique

Automatic short answer grading (ASAG) techniques are designed to automatically assess short answers written in natural language having a length of a few words to a few sentences. In this paper, we report an intriguing finding that the set of short answers to a question, collectively, share significant lexical commonalities. Based on this finding, we propose an unsupervised ASAG technique that only requires sequential pattern mining in the first step and an intuitive scoring process in the second step. We demonstrate, using multiple datasets, that the proposed technique effectively exploits wisdom of students to deliver comparable or better performance than prior ASAG techniques as well as distributional semantics-based approaches that require heavy training with a large corpus. Moreover, by virtue of being independent of instructor provided model answers, our technique offers consistency by overcoming the limitation of undesired variability in performance exhibited by existing unsupervised techniques.